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I f you’re leading but no one is following, you’re simply out for a walk; 
if you’re teaching but no one is learning, are you simply out for a 
talk?” That paraphrase of the author John Maxwell’s observation 
flowed from the lips of my college president with a sheepish grin. 

I decided that to respond to the president’s challenge, I must exam- 
ine the added value of strengths-based interventions. I redesigned my 
dissertation experiment to measure the impact of strengths-based teach- 
ing in two classrooms of our school’s Introduction to Public Speaking 
course. To establish that I wasn’t simply “out for a talk,” I required an 
additional text, implemented new assignments, and devoted the first four 
days of the semester to a strengths-development program in an already 
crowded course curriculum. 

Research Design 

Numerous research studies (Austin 2005; Gillum 2005; Norwood 
2005; Anderson, Schreiner, and Shahbaz 2003, 2004; Turner 2004; 
Williamson 2002) suggest that strengths-development programs posi- 
tively affect many traits: student optimism, strengths awareness, self-con- 
fidence, self-acceptance, goal-directedness, affirmation of others, sense of 
control, realistic expectations, GPA, avoidance of disciplinary action, 
quality of effort, school involvement, timely class attendance, ability to 
effectively collaborate with colleagues, clarity for developing a career 
path and goals, confidence to assume advanced administrative or leader- 
ship roles within public schools, academic self-efficacy, positive self-con- 
cept, positive perception of others, and awareness of others’ strengths. 

No studies, however, have been conducted regarding the impact of 
personal-strengths development on specific learning outcomes. 
Consequently, I designed an experiment in an Introduction to Public 
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Speaking course with student-learning outcomes culled from course syl- 
labi, instructors’ manuals, literature, and the National Communication 
Association. The independent variable was the instructional method — 
either traditional or strengths based. The three dependent variables 
were the content knowledge of the course, measured by average exam 
scores; speech-delivery skills, measured by videotaped speeches; and lev- 
els of academic engagement, measured by the Academic Engagement 
Index (Schreiner 2004). 

Of course, several considerations were critical to ensuring the feasi- 
bility as well as the validity of my proposed experiment. They included 
research design, treatment fidelity, methodological framework, sample, 
instruments, and procedures. 

A quasi-experimental, nonequivalent control-group design that 
employed pre-tests and post-tests was chosen for three reasons. First, 
examining the impact of a strengths-based approach necessitated a con- 
trol group against which to compare the effects of the independent vari- 
able — the instructional method. Second, an inability to assign 
participants randomly to course sections created a need for pre-tests, 
which functioned as covariates to prevent the potential threat that dif- 
ferential selection (age, gender, IQ, reading ability, attitude) could pose to 
internal validity (Shadish, Cook, and Campbell 2002). Third, the design 
offered the most statistical power under these circumstances to explore 
the differences between the two groups and begin to establish a cause- 
and-effect relation between the independent and dependent variables. 

The use of pre-tests prior to the intervention enabled control not 
only for differential selection but also for assessing whether the treat- 
ment and control groups were significantly different from one another 
on variables that would affect their performance on the dependent vari- 
ables. Three pre-test measures were included: (1) participants’ previous 
knowledge of the course content of a public-speaking class; (2) public- 
speaking delivery skills at the beginning of the course; and (3) level of 
academic engagement the first week of class. Because the three vari- 
ables were hypothesized to affect the participants’ performance on the 
dependent variables that were measured at the end of the course, they 
were used in the statistical analyses as covariates. 

Two additional threats to internal validity were considered and con- 
trolled for in this study: the Hawthorne effect and the experimenter 
effect (or experimenter bias). The Hawthorne effect (Gall, GaU, and Borg 
2003), or improvement in participants’ performance simply through 
awareness of their involvement and of the special attention accorded 
their participation, was addressed by minimizing special attention given 
to the research participants. 1 did not purposefully engage the partici- 
pants outside class, except by appointment, or meet with the partici- 
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pants socially, treating both the experimental and control groups equally 
and making sure both groups were aware that they were part of an 
experiment. 

The issue of experimenter bias was important because I alone 
designed and conducted the study, interpreted the data, administered the 
treatment, and collected the data. Therefore, experimenter bias was 
addressed by using treatment-fidelity videotaping (Gall, GaU, and Borg 
2003). I was videotaped each day in both classes. Although each class 
session was videotaped for a total of forty-two sessions, there were a pos- 
sible nineteen regular class days during which I conducted the class for 


the entire fifty-minute session. Other 
delivering speeches, taking exams, 
or listening to a guest lecturer. 

To further ensure treatment 
fidelity, a trained independent rater 
holding faculty status in education at 
her institution was given six ran- 
domly selected videotapes. Each 
videotape contained one day’s ses- 
sions. The viewing of the tapes was 
counterbalanced by giving the rater 
six videotapes: three that began with 
the experimental-group session and 
three that began with the control- 
group session. Each videotape was 
evaluated for uniformity of my class- 
room behaviors and attitudes meas- 
ured by an instrument adapted from 
Assessing Faculty Work: Enhancing 
Individual and Institutional 
Performance (Braskamp and Ory 
1994) after consultation with higher- 
education professionals. 

Sample 

Description 

My sample included sixty first- 
time college freshmen who enrolled 
in one of two Introduction to Public 
Speaking courses. Participants vol- 
untarily enrolled in each section 
without knowing the teaching 
methodology. A coin flip deter- 


class days involved the students 
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mined one section as the experimental class that would be taught from 
a strengths-based perspective. The other section, designated the control 
group, was taught using the traditional method of teaching an 
Introduction to Public Speaking course. Both sections of the course met 
three days each week for fifty-minute sessions across one fourteen-week 
semester. The total class time was forty-two periods, and thirty students 
per class completed the three pre-tests. 

Rationale 

An Introduction to Public Speaking class was the course selected for 
this research study because I had experience teaching public speaking; 
public speaking is a core course at my college. Enrollment was limited 
to entering freshmen because of the importance of the first year in get- 
ting participants a good start in their education (Tinto 1993). 

Dependent Measures 

Measurements of Public-Speaking Content Knowledge and Learning 

Four examinations (one at the beginning of the semester; three dur- 
ing the course of the semester) measured how well students learned 
content elements of effective oral communication measured by average 
exam scores, and thus the dependent variable. Those four examinations 
were objective assessments of basic concepts and principles of oral 
communication and public speaking. Specific items were extracted from 
the instructor’s manual (Lucas 2004) based on the section of the text 
covered and on consultation with other professionals who had advanced 
training in speech and oral communication. 

A twenty-five-item objective pre-test was administered on the first 
day to determine the participants’ pre-existing, discipline-specific knowl- 
edge of public speaking. The other three instruments were objectively 
scored in-class examinations that measured the extent to which students 
learned the content elements of effective oral communication. They cov- 
ered assigned sections of the textbook and my lectures. Those three 
instruments were averaged and used to assess the extent to which stu- 
dents learned the content elements of effective oral communication, 
thus measuring the first dependent variable. 

Measurements of Delivering Ejfective and High-Quality Speeches 

The Competent Speaker Speech Evaluation Eorm, a statistically reli- 
able and valid tool for the assessment of public-speaking performance 
developed by the Speech Communication Association (Morreale et al. 
1993), was administered on two occasions. The form, designed for eval- 
uating public-speaking skills, focuses on assessing the verbal and non- 
verbal behaviors involved in competent public speaking, as opposed to 
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knowledge about, or motivation to engage in, public speaking. The 
instrument has established overall interrater reliability for the partici- 
pants’ total score on the instrument with Ebel’s coefficient reading 0.92. 
In addition to reliability testing, the instrument and its criteria met con- 
tent or face validity (Morley, Morreale, and Hulbert-Johnson 1991). 

For my study, the Competent Speaker Speech Evaluation Form was 
used by independent raters to evaluate the participants’ pre-test and 
post-test videotaped speeches. Each participant gave two live-minute 
speeches to inform, and each one was videotaped. The videotaped 
speeches were used to assess the participants’ speech-delivery skills. The 
first speech (pre-test) was given during the first week of the experiment, 
and the second speech (post-test) was given during the last week of the 
experiment. 

Two communication professionals with graduate degrees and fac- 
ulty rank at their respective institutions received training on the instru- 
ment before interrater reliability was established at 75 percent (Stemler 
2001). The two raters then evaluated fifty-five pre-test and post-test 
videotaped speeches that had been randomly sorted. The blind raters 
knew neither whether the speakers were in the treatment or the control 
group nor whether the speeches were given during the first week or the 
final week of the experiment. The evaluators used clearly defined scores 
on a six-point scale in each of the instrument’s eight competencies. 

Measurements of Levels of Academic Engagement 

The Academic Engagement Index (Schreiner 2004) was adminis- 
tered on two occasions as both a pre-test and post-test measurement of 
the participants’ academic engagement in college. The established coef- 
ficient alpha reliability of this twenty-item instrument is 0.84 (personal 
communication, L. A. Schreiner, June 16, 2004). All twenty questions 
were included as the pre-test, administered during the first week of the 
term, and post-test, administered during the last week of the term. 

Strengths-Based Instruction 

The two sections of the Introduction to Public Speaking course 
were taught by me, but in two distinctive manners. I taught the experi- 
mental group using a strengths-based approach, which was the inde- 
pendent variable. The control group did not receive that treatment. 
Instead, I taught the control group from a traditional method character- 
istic of most public-speaking courses (Lucas 2004). 

The Clifton Strength sFinder 

The independent variable of a strengths-based approach to teaching 
included the use of the Gallup Organization’s Clifton StrengthsFinder 
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(Gallup Organization 1998) to assess participants’ strengths. The experi- 
mental group received that instrument, and the control group did not. 
The StrengthsFinder has an average coefficient alpha reliahility of 0.79 
and six-month test-retest reliahility estimates ranging from 0.60 to 0.80 
across the thirty-four themes (Gallup Organization 2004). For this study, 
the experimental-group participants were introduced to the 
StrengthsFinder after they had completed all pre-tests. The participants 
took the assessment during the first week of the semester. 

Strengths-Based Instruction 

Strengths-based approach. I taught the experimental group from a 
strengths-based method that involved two steps. The first step involved 
identifying and affirming the strengths and talents of each participant by 
administering the StrengthsFinder. The second step involved encourag- 
ing and reinforcing the participants to develop and intentionally apply 
their strengths and talents in learning and performance activities. In four 
fifty-minute class sessions the participants: (1) identified for their peers’ 
strengths discovered through the online assessment, the StrengthsFinder, 
(2) identified at least one strength that they would intentionally use in 
reading a chapter in their public-speaking textbook; (3) identified at least 
one strength that they would intentionally use when studying for an 
examination; and (4) were encouraged to use their strengths more inten- 
tionally and consistently as they learned and performed in the 
Introduction to Public Speaking class (Clifton and Anderson 2002). 

The process of reinforcing participants was an ongoing process of 
interaction between the class and me, both collectively and individually. 
For example, after speeches were given, I called particular attention to 
the point at which each participant performed best. I then helped the 
participants understand how their specific strengths and talents enabled 
them to perform well in that particular aspect of the public-speaking 
process. Further,! encouraged the participants to think of ways in which 
they could use their specific strengths in making their speeches even 
more effective. Participants also were encouraged to think of how they 
could apply their most powerful strengths to other aspects of speech 
delivery in which they wanted to be more effective. 

Traditional approach. I taught the control group using the tradi- 
tional method of professor-centered articulation of course content and 
pubUc-speaking principles. That method of teaching and educating 
involves three basic elements. The first component identified the weak- 
nesses of student performance. The second aspect focused attention on 
the areas that needed improvement. The third element included instruct- 
ing participants in what they needed to learn and how they needed to 
improve to increase their performance. 
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The overt message given to participants by the traditional method 
of teaching was that if they wanted to achieve, then they had to over- 
come their weaknesses and areas of underperformance. Although some 
class sessions were devoted to methods of overcoming specific areas of 
weakness, I focused the participants’ attention on specific areas of their 
speech preparation and performance that needed improvement. Similar 
patterns were followed in giving feedback on tests and other written 
assignments. 

Treatment Fidelity 

The ten-item Classroom Observation Rating Form measured and 
assessed my integrity in administering the intervention. Specifically, this 
instrument measured the uniformity of my classroom behaviors in both 
the experimental and control groups. The form was adapted from the 
work of Braskamp and Ory (1994) after consultation with other higher- 
education professionals. Six videotaped sessions that included both 
groups were randomly selected. The videotapes were then counterbal- 
anced: three were wound to begin with the experimental group, and 
three were wound to begin with the control group. The six videotapes 
were rated by an education professional with faculty rank at her institu- 
tion; she was paid to serve as a blind rater. 

Conclusion 

Often, irony occurs in conducting research investigations. 
Sometimes the very best discoveries and the very best insights are not 
captured by the dependent-variable measurements established at the 
beginning of the experiment. Only one week after beginning the exper- 
iment, I began documenting behavior patterns of the students in both 
sections. There were enormous differences. Students in the strengths 
class typically came to class on time, while students in the traditionally 
taught class did not. Students in the strengths-based class had better 
class attendance, while students in the traditional class missed class 
more frequently. In the traditional class, I frequently had to stop teach- 
ing in order to curtail side conversations and disruptive behavior. 
However, I rarely had to say anything about side conversations or dis- 
ruptive behavior in the strengths class. There was an average of three 
times more questions asked and three times more contributions made to 
discussions in the strengths class. Those patterns also were evident in 
whether assignments were turned in on time. All such patterns are 
direct indicators of academic engagement and are supported by the lit- 
erature on behavioral academic engagement (Birch and Ladd 1997; 
Fincham, Hokoda, and Sanders 1989). 
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I was now ready to approach my college president and tell him that 
I simply was out for neither a walk nor a talk; students were learning and 
I could prove it! Research would suggest that we cannot overemphasize 
the importance of engaging students academically. Winston Doby (as 
cited in Clifton and Anderson 2002) has asserted that the fundamental 
challenge of every educator is how to engage the mind of the learner. 
Simply stated, if the mind of the learner is not engaged, learning will not 
occur (Brophy and Good 1986; Connell, Spencer, and Aber 1994; Connell 
and Wellborn 1991; Finn 1993; Finn and Rock 1997;Hofer 2002). 
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